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OBJECTIVE — Inexpensive and standardized methods to deliver medical education to pri- 
mary care physicians (PCPs) are desirable. Our objective was to assess the impact of an individ- 
uahzed simulated learning intervention on diabetes care provided by PCPs. 

RESEARCH DESIGN AND METHODS — Eleven clinics with 41 consenting PCPs m a 
Minnesota medical group were randomized to receive or not receive the learning intervention. 
Each intervention PCP was assigned 12 simulated type 2 diabetes cases that took about 15 min 
each to complete. Cases were designed to remedy specific physician deficits found in their 
electronic medical record observed practice patterns. General linear mixed models that accom- 
modated the cluster randomized study design were used to assess patient-level change from 
preintervention to 12-month postintervention of AlC, blood pressure, and LDL cholesterol. The 
relationship between the study arm and the total of intervention and patient health care costs was 
also analyzed. 

RESULTS — Intervention clinic patients with baseline AlC ^7% significantly improved gly- 
cemic control at the last postintervention AlC measurement, intervention effect of —0.19% 
mean AlC (P = 0.034) and +6.7% in AlC <7% goal achievement (P = 0.0099). Costs trended 
lower, with the cost per patient —$71 (SE = 142, P = 0.63) relative to nonintervention clinic 
patients. The intervention did not significantly improve blood pressure or LDL control. Models 
adjusting for age, sex, and comorbidity showed similar results. PCPs reported high satisfaction. 

CONCLUSIONS — A brief individualized case-based simulated learning intervention for 
PCPs led to modest but significant glucose control improvement in adults with type 2 diabetes 
without increasing costs. 



Although continuing medical educa- 
tion (CME) is required for physi- 
cian licensure in many states, there 
is a paucity of evidence that traditional 
methods change physician behavior or 
improve clinical care (1). The Association 
of American Medical Colleges has called 
for increased study of CME approaches 
that incorporate adult learning principles, 
including tailored curricula and interac- 
tive experiences (e.g. , simulated activities, 
self-directed learning, immediate feed- 
back) (2). Of particular importance is em- 



Diabetes Care 33:1727-1733, 2010 

phasis on case -based context-dependent 
learning situations that address the com- 
plexity of specific clinical domains and fo- 
cus on transfer of knowledge and skills 
beyond the specific learning situation (3). 
Personalized outreach visit approaches 
such as opinion leader interventions and 
academic detailing may influence some 
aspects of clinical care (4,5), including in- 
creased use of aspirin and (5-blockers for 
acute myocardial infarction (5), antibiotic 
choices (6), and HIV prevention practices 
(7). However, associated drawbacks in- 



clude costs, teaching nonuniformity, and 
difficulty finding quality opinion leaders 
in nonacademic settings. 

Simulated case-based learning offers 
an efficient and less expensive personal- 
ized physician learning alternative. It has 
been successful in aviation and chemical 
engineering and is increasingly used in 
health care training programs (8,9) to 
teach Advanced Cardiac Life Support (8), 
various surgical skills (10), and airway 
management (11). Furthermore, our pre- 
viously published evaluation of a brief 
prototype of the learning intervention 
used in this trial was encouraging (12). 
Therefore, development and rigorous 
evaluation of simulation approaches to 
teach the cognitive skills of chronic dis- 
ease care management are warranted. 

RESEARCH DESIGN AND 
METHODS 

Hypothesis 

This group-randomized trial was de- 
signed to test whether a personalized 
simulated learning intervention for phy- 
sicians would improve care delivered to 
adults with uncontrolled diabetes. 



Study setting, subjects, and design 

The study was conducted from October 
2006 to May 2007 at HealthPartners 
Medical Group (HPMG), a large medical 
group in Minnesota that serves about 
230,000 patients. Eleven HPMG clinics 
were randomly selected and block ran- 
domized on the basis of baseline quality of 
diabetes care and number of consenting 
primary care physicians (PCPs) to either 
receive or not receive the intervention. 

PCPs were eligible for the study if 
they practiced in one of the study clinics, 
provided care to at least 10 adult patients 
with diabetes, and signed a consent form. 
Patients were classified as having diabetes 
if they had two or more outpatient diabe- 
tes ICD-9 codes or used a diabetes- 
specific medication in the year before 
randomization (see Fig. 1 for the detailed 
consort description). 
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11 clinics and 
86 PCPs 



6 CLINICS ALLOCATED TO INTERVENTION 
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- 5 PCPs did not meet inclusion criteria* 
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Included in the analysis: 
5 clinics 
20 PCPs 

1 ,570 patients with diabetes linked to study PCPs, 
with exclusions (or: 

-566 age out of range 

-270 not consistently assigned to provider 

-32 patients died 



Figure 1 — Diagram illustrating allocation of clinics, PCPs, and diabetes patients to the two study arms. The diagram also shows the disposition of 
diabetes patients who were and were not included in the analysis. *Eligible PCPs worked >60% of a full time equivalent and had ^10 patients with 
diabetes. * *Patients were linked to the study-consented last-assigned PCP during the preintervention period and study -consented first-assigned PCP 
during the postintervention period. Patients lacking an assigned PCP were linked to the provider seen most during the study period. PCP, primary 
care physician. 



Intervention design 

A group of PCP experts actively involved 
in diabetes guideline development identi- 
fied and reached consensus on 25 essen- 
tial clinical practices for the successful 
management of type 2 diabetes in adults 
(Table 1). The goal of the learning pro- 
gram was to promote mastery of these es- 
sential clinical practices. 

The research team then used existing 
electronic medical record data to profile 
individual PCP performance on the essen- 
tial clinical practices. If a physician per- 
formed below average compared with his 
or her physician peers on a cHnical prac- 
tice, a simulated case initialized with clin- 
ical parameters designed to teach the 
specific clinical practice was assigned. 

Initialization parameters for the set of 
learning cases were defined to represent the 
wide range of complex clinical scenarios a 
PCP could encounter. Each case encom- 
passed a different mix of baseline patient 
demographics (age, sex, duration of diabe- 



tes), medical history and comorbidities 
(congestive heart failure, renal insuffi- 
ciency, coronary artery disease), pharmaco- 
logic use (active medications for glycemia, 
blood pressure [BP] , lipids, depression, as- 
pirin), clinical states (AIC, BP, lipid, creat- 
inine, and self-monitoring of blood glucose 
results), and other subjective patient char- 
acteristics (hypoglycemia symptoms, med- 
ication adherence, lifestyle habits, and 
depressive symptoms). An automated case 
generator was developed to create distinct 
simulated learning cases. 

A detailed description of the simulation 
software has been previously published 
(13). The patient model embedded in the 
software uses prespecified formulas derived 
from dose-response curves for drugs, life- 
style advice, and referrals. This allowed cal- 
culated changes in the patient state at each 
encounter on the basis of physician treat- 
ment actions. The interface mimics elec- 
tronic medical record screens that permit 
the learner to prescribe drugs, order labs or 



diagnostic tests, make referrals, give patient 
advice, change frequency of recommended 
self-monitoring of blood glucose testing, 
view self-monitoring of blood glucose re- 
sults, start or adjust insulin with each meal 
and at bedtime, and see the patient at any 
desired frequency for phone or office visits. 

A key strategy of the learning interven- 
tion was the individualization embedded in 
the intervention at multiple levels. First, the 
PCPs received a customized set of simulated 
cases selected to address their assessed 
learning needs. Second, feedback resulted 
from a multitude of individual provider ac- 
tions, yielding a unique trajectory to each 
case in response to the learner's specific 
treatment decisions. Learning feedback oc- 
curred through seeing the clinical effects of 
treatment moves at subsequent encounters, 
seeing graphic displays of the projected re- 
sults of accumulated treatment actions, and 
direct textual feedback after each encounter 
consisting of a critique of past actions and 
suggestions for future ones. 
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Table 1 — The 25 essential clinical care practices taught in the simulated program 



Sperl-Hillen and Associates 



Glycemia practice 



Hypertension practice 



Lipid practice 



1 Early drug initiation after medical nutrition 

therapy failure 

2 Initiation of additional oral drugs or 

exenatide beyond metformin and 
sulfonylureas 

3 Initiation of insulin or exenatide 

4 Change to updated insulin regimen (basal 

bolus insulin regimens) 

5 Initiation of metformin as insulin sensitizer 

6 Initiation of thiazolidinedione as insulin 

sensitizer 

7 Initiation of prandial insulin 

8 Titration of basal insulin 

9 Titration of prandial insulin 

10 Titration of insulin in large enough 

amounts 

1 1 Use of SMBGs and pattern recognition 

12 Optimal metformin dosing 

13 Optimal sulfonylurea dosing 

14 Optimal insuUn dosing 

15 Optimal thiazoUdinedione dosing 

16 Timely visit intervals 

17 Avoid severe hypoglycemia (<60 mg/dl) 

18 Address mild hypoglycemia (60-69 mg/dl) 

19 Avoid fear of low normal glucose levels 

(70-89 mg/dl) 

20 AIC frequency ^3 months if not at goal 

21 AIC frequency ^6 months at goal 

22 Metformin/creatinine contraindications 

23 Metformin/heart failure warnings 

24 Thiazolidmedione/heart failure warnings 

25 Educator referrals for patients not at goal 



Initiation of BP medication, new diagnosis 
Appropriate use of home BP measurements 



Initiation of combo drugs for stage 2 hypertension 
Adding drug classes as needed 



Initiation of fourth dru^ 
Titration of drugs 



class 



Target systoUc BP 

Importance of treatment in the elderly 
More frequent visit intervals for patients not at 
goal 

Monitoring potassium and creatinine after starting 
or increasing an ACE inhibitor or angiotensin 
receptor blocker 

Use of ACE inhibitor or angiotensin receptor 
blocker for congestive heart failure 



Initiation of statin above goal 
Use of fibrate for high triglyceride 



Use of fibrate for low HDL 

Titrating statin or ezetimbe to achieve 

LDL goals 
Titrating lipid drugs, fear of myalgias 
Reassessing lipids ^3 months after 

adjusting lipid medications 
Yearly monitoring of lipids 
Monitoring liver enzymes tests 



MNT, medical nutrition therapy; TZD, thiazolidinedione; HDL, liigli-density lipoprotein cholesterol; LDL, low-density lipoprotein cholesterol; SMBG, self- 
monitored blood glucose; ACE, angiotensin-converting enzyme inhibitor; ARB, angiotensin-receptor blocker; TG, triglyceride. 



Intervention implementation 

PCPs were sent an email link to the pro- 
gram, accessed through a secure login 
page housed on an internal server. Click- 
ing on the name of one of the 12 assigned 
cases started the simulation. The physi- 
cian was challenged to achieve the Amer- 
ican Diabetes Association-recommended 
treatment goals for BP, lipids, and AIC 
within 6 months of simulated time. Each 
case took about 15 min to complete. 

Physicians received financial com- 
pensation ($600) and up to 6 h of CME 
credit for completing the program, which 
had to be done on their own time. The 
average number of days to complete the 
12 cases was 5.5. At the end of the inter- 
vention, physicians also were sent an eval- 
uation of the learning program. 



Dependent variables 

Measures of intermediate outcomes 
(AIC, BP, LDL levels) of diabetes care in 
patients of providers extracted from the 
electronic medical record were the prin- 
cipal dependent variables. The baseline 
measurements were the clinical values 
viewable by the provider at the first en- 
counter after completing the learning in- 
tervention. For AIC and LDL, this was the 
last clinical value obtained in the 1-year 
preintervention period. Because of point- 
of-care availability of blood pressure 
levels at encounters, the baseline mea- 
surement for BP was the first value in the 
1-year postintervention period. For all 
measures, the last value in the postinter- 
vention period is selected as the post- 
intervention variable. No changes in 



laboratory assay methods occurred dur- 
ing the study. 

Costs were estimated from the health 
plan perspective and included the costs of 
the intervention and health care costs. In- 
tervention costs included J ) marketing to 
physicians; 2) profiling physician learn- 
ing needs; 3) implementing physician 
training, case assignment, and email re- 
minders; and 4) physician compensation 
for completing cases. Health care costs in- 
cluded outpatient services and pharma- 
ceuticals. Inpatient services were not 
included because the intervention was 
not expected to affect hospitalization 
rates. Costs were estimated using meth- 
ods previously developed to assign costs 
of services in this medical setting (14). 
Services were based on relative value 
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Table 2 — Characteristics of study physicians and diabetes patients linked to those study 
physicians at intervention and control clinics 



units assigned on the basis of procedure 
codes recorded and priced at the 2009 
physician services conversion factor of 
$36.07. Costs of pharmaceuticals were 
based on 68% of the average 2009 whole- 
sale price. 

Independent variables 

The independent variable was an indica- 
tor for study arm. The interaction of this 
with time was used to assess the differen- 
tial impact of the intervention on the pre- 
specified outcomes. Because the trial was 
group-randomized at the clinic level, im- 
balance in patient characteristics was ex- 
pected. Patient-level independent 
variables include age, sex, race, and vali- 
dated indicator variables for coronary ar- 
tery disease and congestive heart failure. 
Adults older than 80 years and individu- 
als with Charlson comorbidity scores of 
&3 (indicating high short-term risk of 
mortality) were excluded from the study 
because of legitimate debate about appro- 
priate clinical goals in such scenarios 
(15). 

Statistical analysis 

Attributes of study-consented physicians 
and patients linked to these physicians are 
compared by study arm using descriptive 
statistics (mean, SD, proportions) and us- 
ing independent sample £ tests, contin- 
gency tables, and Pearson tests. 

General and generalized linear mixed 
models with a repeated time measure- 
ment (baseline and postintervention) 
were used to analyze continuous and bi- 
nary outcomes using SAS Proc Mixed and 
Proc Glimmix. These models included a 
term for study arm, time (baseline or 
postintervention) , a time by study arm in- 
teraction term, and random intercepts to 
account for multiple levels of nesting. The 
time-by-study-arm interaction term 
tested the effect of the intervention arm 
over time relative to the effect of the con- 
trol arm over time. The analyses on test 
values were also conducted predicting 
postintervention values from study arm, 
preintervention test value, and patient 
covariates. 

Denominators for the analysis of test 
rates, encounter rates, and numbers of 
tests and encounters include all (n = 
3,417) eligible patients linked to study- 
consenting physicians. Analyses for 
change in values (e.g., AlC) are based on 
subsets of patients who are not at goal at 
baseline on particular measures of inter- 
est, because they are targeted in the inter- 
vention. A priori sample size calculations 



Patients (n) 
Age (years) 
Female (%) 
White race (%) 

Coronary artery disease during 

preintervention (%) 
Congestive heart failure during 

preintervention (%) 
Preintervention first AlC value 
Median 

Preintervention first SBP value 
Median 

Preintervention first DBP value 
Median 

Preintervention first LDL value 
Median 
PCPs (n) 
Age (years) 

Family practice physicians (%) 
Female (%) 

Number of adult diabetes patients per PCP 



assumed an analytic sample of 500 dia- 
betic patients per study arm, based on 20 
providers with 25 diabetic patients not at 
AlC goal. Effective patient sample size 
was estimated as n = 291 per arm due to 
clustering of patients within physicians 
(estimated intraclass correlation [ICC] = 
0.03). This study was designed with 80% 
power to detect an AlC difference of 
0.3% between study arms, with a two- 
tailed a = 0.05. Alpha levels are not ad- 
justed for testing of multiple dependent 
variables. 

Generalized linear models, assuming 
7 distribution and a log link function, 
were used to analyze the relationship be- 
tween costs and study arm (16). These 
models included a term for study arm, 
time (baseline or postintervention), and a 
time-by-study-arm interaction term. A 
standardized estimate of the effect of the 
intervention on costs was calculated as a 
mean difference in predicted costs among 
study patients (including all patients with 
AlC >7% in the preintervention period), 
as they were alternatively assigned to the 
intervention and control groups in the 
post period. SEs were estimated using 
the nonparametric bootstrap, and signifi- 
cance values were computed using the 
percentile method (17). 
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Protection of human study subjects 

The study was reviewed in advance, ap- 
proved, and monitored on an ongoing ba- 
sis by the HealthPartners Institutional 
Review Board, Project #03-083. 

RESULTS— Attributes of study- 
eligible patients and PCPs are presented 
in Table 2. Diabetic patients' age was 
56.4 ± 10.7 years (mean ± SD), 24.0% 
were 65-74 years old, 48.9% were fe- 
male, and 29.2% were nonwhite. At base- 
line (first preintervention value), 47.5% 
had AlC <7%, 61.7% had systolic BP 
(SBP) <130 mmHg, 67.3% had diastolic 
BP (DBP) <80 mmHg, and 60.2% had 
LDL < 100 mg/dl. The number of diabetic 
patients per study-enrolled PCP ranged 
from 10 to 125, with a mean of 56.4 ± 
29. 1. Randomization at the clinic level re- 
sulted in an intervention arm with a 
higher proportion of younger and male 
patients. 

In four-level random intercept mod- 
els (measurement occasion nested within 
patient, provider, and clinic), ICCs at the 
clinic level were generally small, with val- 
ues of ICC <0.0001 for AlC and LDL 
value, ICC = 0.001 for SBP value, and 
ICC = 0.003 for DBP. Because of the low 
level of variance at the clinic level, three- 



Data are means ± SD unless otherwise indicated. 'Independent samples £ test or Pearson x^- CHF, congestive 
heart failure; LDL, low-density lipoprotein cholesterol; PCP, primary care physician; DBP, diastolic blood 
pressure; SBP, systolic blood pressure. 
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level models are presented by dropping 
the random intercept term for clinic. 

Table 3 shows baseline and follow-up 
measures of diabetes encounters, AlC, 
BP, and LDL test rates. They increased be- 
tween pre- and postintervention mea- 
surements, with no differential effect for 
the intervention group, as shown by the 
nonsignificant time-by-condition interac- 
tion term P values. 

Statistically significant improvements 
over time were seen within study arm for 
AlC, SBP, DBP, and LDL values. Inter- 
vention-arm patients had a significantly 
greater improvement (decrease) in AlC 
value from baseline to postintervention 
than the control arm by a value of 
-0.19% (95% CI -0.37 to -0.01, P = 
0.034). Among patients with an AlC 
£7% at the last preintervention measure- 
ment, 29.2% of intervention-arm and 
22.5% of control-arm patients had an 
AlC <7% at the last postintervention 
measurement (P = 0.0099, n = 1,403). 
There were no significant differences in 
SBP or DBP reduction over time by study 
arm. Control-arm patients had a signifi- 
cantly greater improvement (decrease) in 
LDL value from baseline to postinterven- 
tion than the intervention arm by a value 
of -5.1 mg/dl (95% CI -9.8 to -0.3, 
P = 0.039). Analysis of the proportion of 
patients at LDL goal (< 100 or <70 mg/dl 
with coronary artery disease) at the last 
measurement in the 12-month postinter- 
vention period showed no significant dif- 
ference (0.413 intervention clinics, 0.414 
control clinics, P = 0.99, n = 1,069). 

Intervention costs were estimated to 
be $27 per patient. Total costs, including 
intervention and health care costs, were 
estimated to be $71 (SE = 142, P = 0.63) 
lower per patient in the intervention clin- 
ics compared with control clinics. 

A total of 85% of intervention PCPs 
completed a survey of satisfaction and 
self-assessed impact of the learning. Of 
these, 88% would recommend the learn- 
ing experience to other physicians, 82% 
thought it would help most doctors im- 
prove their diabetes care skills, 82% re- 
ported they would be more likely to 
intensify medication for their diabetic pa- 
tients, 59% would shorten their visit in- 
tervals, and 18% would start insulin more 
often. A separate analysis showed that 
preintervention PCP performance mea- 
sured by the percentage of patients at gly- 
cemic goal was not a significant predictor 
of AlC postintervention (P = 0.98). 



CONCLUSIONS— This personal- 
ized physician learning intervention dem- 
onstrated a modest but significant AlC 
lowering without increasing patient visits 
or total net costs. The observed AlC im- 
pact (a 0.5% improvement in AlC in the 
intervention group, which was 0.19% 
better than the control group) is of 
roughly the same magnitude as that re- 
ported in uncontrolled observational 
studies of more expensive interventions 
intended to improve diabetes care, such 
as clinical information systems (18), pa- 
tient education, and disease management 
programs (19). The net AlC improve- 
ment is clinically significant based on the 
potential to reduce patient complications, 
as demonstrated through UK Prospective 
Diabetes Study results showing 37% 
lower microvascular complications for 
every 1% absolute AlC reduction (20). 

PCPs reported high levels of satisfac- 
tion with the intervention, repeated learn- 
ing cases voluntarily, and reported 
tangible changes in the way they manage 
diabetes. The learning effect of the com- 
puter-based intervention was present in- 
dependent of PCP baseline diabetes 
performance and indicates an ability to 
transfer what was learned from the simu- 
lated cases to the care of real patients — an 
important challenge and desirable finding 
in simulation research. 

There are a number of points to make 
about the cost of this intervention in rela- 
tion to its modest clinical effectiveness. 
First, the intervention did not increase pa- 
tient visits or testing rates in real patients 
and trended toward cost-saving. Second, 
this simulated approach to physician 
learning is compatible with a wide range 
of chronic care improvement activities 
designed to activate patients or develop 
prepared care teams (21). Finally, the 
simulation content can be easily adapted 
to accommodate changes in care guide- 
lines, discourage unnecessary tests or 
treatments, and encourage use of generics 
and might help modulate to some degree 
the high costs of diabetes care. The mar- 
ginal costs for ongoing use of this learning 
tool are relatively small and principally 
involve periodic updates to ensure that 
the simulation model remains current 
with evidence-based treatment strategies 
and newly approved drugs. 

Future research should investigate 
issues that might increase the impact, 
reduce the complexity, or broaden the 
dissemination potential of the interven- 
tion. Our use of electronic medical re- 
cord-based physician treatment 



patterns to assess learning needs is in- 
novative but can be difficult. Develop- 
ment of simpler methods to evaluate 
physician learning needs (e.g., assess- 
ment on a set of prespecified simulated 
cases) warrants investigation. Second, 
the intervention needs adaptation for 
web distribution and to simplify addi- 
tion of clinical updates when needed. 
Third, at the time of this study, gener- 
alized diabetes care goals, such as AlC 
of <7%, were widely accepted. As per- 
sonalized medicine seeks to accommo- 
date individualized treatment goals, 
this approach provides an opportunity 
to teach a systematic approach to setting 
clinical care goals individualized to patient 
characteristics such as comorbidities, 
polypharmacy concerns, hypoglycemia 
risk, and genetics (22,23). 

Additional work is needed to eluci- 
date more precisely the mechanisms re- 
sponsible for the observed effects. The 
variable effect on clinical domains may be 
partially related to primary emphasis on 
glucose-related feedback in the learning 
cases relative to BP or lipid-related feed- 
back. Recent clinical trial results (23,24) 
highlight the importance of directing 
more attention to BP- and lipid- 
management issues. It is notable that the 
study site had relatively good baseline lev- 
els of AlC, BP, and LDL (25). The impact 
of this learning intervention in settings 
with worse baseline levels of diabetes care 
remains to be determined. This promising 
care-improvement strategy might also be 
applied to other clinical domains. 

Despite some limitations, these data 
demonstrate that delivery of a brief and 
relatively inexpensive individualized 
physician learning intervention improved 
intermediate outcomes of diabetes care 
without increasing costs. Experimenta- 
tion with simulated personalized learning 
interventions in a broad range of other 
care or educational settings seems 
warranted. 
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